Systems and methods for compressive image sensor techniques utilizing sparse measurement matrices

ABSTRACT

Systems and methods for compressive image sensor techniques based on sparse measurement matrices are disclosed. A method to perform compressive sensing (CS) measurement operations for image sensors limits pixel summation to be within neighboring pixels and hence dramatically simplifies CS image sensor circuits and reduces their power consumption while providing better image quality compared to conventional random measurement matrix based methods. A sparse measurement matrix is applied to pixel data to generate a desired number of summation groups, each summation group consisting of outputs from an equal number of pixel cells. Each pair of summation groups contains the same number of shared outputs from pixel cells. From the summation groups, an image captured by the pixel cells is recovered.

CROSS REFERENCE TO RELATED APPLICATIONS

This is a non-provisional application that claims benefit to U.S.provisional application Ser. No. 62/384,510 filed on Sep. 7, 2016, whichis incorporated by reference in its entirety.

GOVERNMENT SUPPORT

The invention was made with government support under grants NSF IIP1535658 and NSF IIP 1361847 both awarded by National Science Foundation.The government has certain rights in the invention.

FIELD

The present disclosure generally relates to compressive image sensortechniques and in particular to systems and methods for compressivesensing image sensor techniques utilizing a sparse measurement matrix.

BACKGROUND

As cameras and camera sensors become increasingly ubiquitous, there areincreasing demands for low-power and high-resolution image sensors. Forexample, such devices are extremely desirable for hand-held or wearablegadgets, and might be mandatory in swallowable medical devices due topower and heat dissipation constraints. Over the past several decades,remarkable progress in image sensor power reduction and resolutionimprovement has been achieved by exploiting novel circuit techniques andutilizing increasingly advanced fabrication technologies. Nevertheless,the image capture and processing flow has largely remained the same. Theimage information is first captured by sensor pixels in an analogformat. Then, each pixel output is converted into digital data by analogto digital converters (ADCs). Thereafter, the digital data iscompressed, processed or transmitted. The ADC operation is relativelypower hungry in the image capturing process. As such, as the number ofADC operations has increased dramatically with the relentlessimprovement of image resolution, further reducing the power consumptionof high-resolution image sensors has become increasingly challenging.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing and other objects, features, and advantages of the presentdisclosure set forth herein should be apparent from the followingdescription of particular embodiments of those inventive concepts, asillustrated in the accompanying drawings. Also, in the drawings the likereference characters refer to the same parts throughout the differentviews. The drawings depict only typical embodiments of the presentdisclosure and, therefore, are not to be considered limiting in scope.

FIGS. 1A-1E illustrate prior art Compressive Sensing (CS) image sensorcircuits, according to aspects of the present disclosure;

FIG. 2 depicts an illustrative example of a CS measurement pattern,according to aspects of the present disclosure;

FIG. 3 presents a comparison of image qualities obtained with theproposed sparse measurement matrices and conventional random matricesfor 1000 benchmark images;

FIG. 4 depicts a circuit for a CS image sensor based on a sparsemeasurement matrix, according to aspects of the present disclosure;

FIG. 5 depicts a circuit for reading pixel outputs and performing pixelsummations in a CS measurement operation, according to aspects of thepresent disclosure;

FIG. 6 depicts control signals during CS image sensor readout operation,according to aspects of the present disclosure;

FIG. 7 presents comparison between the present system and existing CSimage sensor circuits, according to aspects of the present disclosure;

FIG. 8 depicts key component values used in the CS image sensor;

FIG. 9 depicts amplifier performance parameters used in the CS imagesensor; and

FIG. 10 depicts reconstructed Lena and Cameraman benchmark images fromMatlab and circuit simulations, according to aspects of the presentdisclosure.

Corresponding reference characters indicate corresponding elements amongthe view of the drawings. The headings used in the figures do not limitthe scope of the claims.

DETAILED DESCRIPTION

Recently, Compressive Sensing (CS) has emerged as a promising techniqueto address the need for low-power, high-resolution image sensors.Instead of digitizing every pixel, a CS image sensor only digitizes asmall set of random pixel summations. The process to obtain the randompixel summations is often referred to as a CS measurement operation andthe total number of measurements required can be significantly smallerthan the total number of pixels present. From this small set ofmeasurement data, the image can still be reconstructed with highfidelity via CS techniques, providing an interesting paradigm to reduceADC operations as well as power consumption in image sensor circuits,such as Complementary Metal-Oxide-Semiconductor (CMOS) image sensorcircuits.

For purposes of explanation, assume that pixel data is denoted by a onedimensional vector x with N elements. (Although images are twodimensional, the pixel values can be rearranged into one dimensionalformat for the convenience of discussion.) Then, the CS measurementoperation can be described by a matrix operation φ·x, where φ is ameasurement matrix with M rows and N columns, with M<N. At present,existing CS image sensor circuits use dense random measurement matrices,which require complicated circuit implementation and pose stringentsignal swing requirements.

A number of CS image sensors have been reported in the literature. Someof them perform CS measurement operations in the optical domain beforethe image is captured by pixel cells. This helps reduce the number ofpixels on the sensing device and can be desirable in applications whereit is expensive to have a large number of pixels on the sensors, e.g,infrared sensors. Other CS image sensors perform CS measurementoperations on pixel cell outputs. Such sensors are often referred to ascircuit-based CS image sensors, and are designed mainly to reduce sensorADC operations and hence the power consumption of the sensor.

At present, circuit-based CS image sensors can be mainly differentiatedby their technique of operation in performing a CS measurement. FIGS.1A-E depict various circuit-based CS image sensors and measurementprocesses and operations. FIG. 1A illustrates a random measurementprocess which uses a computational pixel design, wherein differentialcurrent output is proportional to the product of the projected lightintensity and its row drive voltage. The current outputs of the pixelcells within the same column are summed at the bit lines and the rowdrive voltages serve as the weight factors in the summations. The columnoutputs are further processed by an analog vector matrix multiplier(VMM) to produce the CS measurement data set. To simplify the circuitimplementation, other CS measurement circuits limit the weight factorsto 1 and −1 in the summation.

Another approach embeds shift registers into the pixel array as shown inFIG. 1B. Random bit streams generated by linear feedback shift registers(LFSRs) are shifted into the array before taking the measurement. If theregister bit in a pixel cell is 1, the current output of the pixel isdirected to positive bit line I_(i−); otherwise, it is connected tonegative bit line I_(j−). The currents from the positive and negativebit lines are subtracted to generate the measurement results. However,embedding D flip-flops (DFFs) into pixel cells significantly increasesthe size of pixel cell circuits. Also, summing a large number of pixeloutputs poses stringent signal swing requirements.

Another existing design is shown in FIG. 1C, which limits the summationoperation for pixels within the same column, partially lessening thesignal swing challenge. It still follows the random summation patternsand uses complicated pixel cell design. Conventional compact pixelcells, e.g. 3-transistor (3T) or 4T pixel designs have also been used inCS image sensor circuits. These designs partition the pixel array intoblocks of 16×16 pixels and limit the random summations for pixels withinthe blocks. Also, large LFSRs are used to generate random bit patternsto guide the summations. These designs limit the measurement matrixelements to 1 and 0. If a matrix element is 1, the corresponding pixeloutput is included in the summation; if it is 0, the corresponding pixeloutput is excluded from the summation.

A design as shown in FIG. 1D uses charge amplifiers to perform pixelsummations. The fan-in limitation of the charge amplifier as well as theamplifier output swing limit the block size. Another existing design asshown in FIG. 1E integrates the summation function into ΔΣ circuits,feeding the selected pixel signals to the ADC input in a round-robinmanner during the ΔΣ modulation process. Large multiplexer circuits areused to route pixel outputs to the ΔΣ input. Overall, existing CS CMOSimage sensors use random measurement matrices for pixel summations. Theytypically require large LFSRs to generate the random bit patterns andutilize complicated circuits to implement the random summationoperation. These factors adversely affect the scalability and the powerefficiency of existing CS image sensor circuits.

It is with these observations in mind, among others, that variousaspects of the present disclosure were conceived and developed.

Theory of Compressive Sensing

CS techniques originate from an interesting mathematical question.Suppose vector x has N elements and satisfies the sparsity condition oforder k (a vector is k-sparse if it has at most k non-zero orsignificant elements). Is it possible to recover vector x from M linearobservations (with M<N)? The linear observations or measurements can bedescribed as matrix operation y=A·x, where A is a matrix with M rows andN columns. In general, solving for N unknown variables from M equations(M<N) is not well-posed and there is no unique solution. However, bytaking advantage of x being k-sparse, and if matrix A meets certainrequirements, it is possible to recover x with high confidence level andgood accuracy. Several conditions for selecting matrix A to guaranteethe recovery of x have been derived by experts in mathematical fields,such as constraints in terms of spark, coherence, null space, restrictedisometry property (RIP), etc. Meanwhile, various methods to recover xhave been also developed, including adaptive binary search, l_(l)minimization (or basis pursuit), greedy pursuits, etc. Among them, theRIP condition and the basis pursuit recovery method are frequently usedin CS related applications. The RIP was originally defined as follows.Matrix A obeys the RIP with constant δ_(k) if:

(1−δ_(k))∥x∥ ₂ ² ≦∥A·x∥ ₂ ²≦(1+δ_(k))∥x∥ ₂ ²   (1)

for all k-sparse vectors x, with μx∥₂ ² denoting the standard l₂-norm on

^(d). To be able to recover vector x, δ_(k) needs to be smaller thancertain thresholds. Intuitively, δ_(k) indicates how well the linearobservations, (i.e. A·x), preserve the energy of the vector or signal x.The smaller δ_(k) is, the better the signal energy of x is preserved.Later, the RIP is generalized with the following condition:

(1−δ_(k))∥x∥ _(p) ² ∥A·x∥ _(p) ²≦(1+δ_(k))∥x∥ _(p) ²   (2)

This condition (2) shows that a RIP with p=1, denoted as RIP-1, can alsobe used to select a matrix A to guarantee the recovery of x. Note thatthese conditions discussed above are typically sufficient, but notnecessary. For example, a matrix that satisfies RIP-1 guarantees signalrecovery, but may not satisfy the original RIP-2 condition, and viceversa. Thus neither of the conditions is stronger than the other.

In real world applications, many signals are not sparse in theiroriginal form, but become sparse after being projected into anotherdomain. The projection operation can be described by x=ψ·c, where x isthe original signal, ψ is the representation or sparse basis, and c isthe coordinate vector of x with respect to basis ψ. For example, asingle-tune sinusoidal signal is not sparse in the time domain but itsFourier coefficient vector is very sparse, containing only one element.Image signals and many biomedical signals manifest similar behaviors.For these types of signals, CS techniques can recover an approximationof the coordinate vector c from only a small set of measurement data.Once c is known, the underlying signal can be easily computed.

Following the above principles, a CS image sensor generates a small setof linear observations by y=φ·x, where x is the vector of pixel data andφ is the measurement matrix. Since the size of the linear observations ycan be dramatically smaller than that of pixel data x, ADC operations aswell as sensor output data can be significantly reduced. At thereceiving end, c can be recovered by solving the following l₁minimization problem:

min∥c∥ ₁, subj.to y=φ·ψ·c   (3)

Thereafter, the pixel data x can be reconstructed by x=ψ·c. Note thatthe product of φ·ψ is the matrix A used in the mathematical formulations(1) and (2) above. Often, a random matrix is used as measurement matrixφ, since it has been shown that in the random case, the product φ·ψsatisfies the RIP or other CS signal recovery constraints with highprobability.

Measurement Technique

Existing CS image sensor circuits use random measurement matrices φbecause the product of the random measurement matrix φ and the sparsebasis matrix ψ is likely to satisfy the RIP constraints. Images aregenerally sparse with respect to several bases, such as the discreteFourier basis and the inverse discrete cosine transform (IDCT) basis, asubgroup of Fourier basis. The vectors in IDCT correspond to samples ofthe cosine function with variable frequency starting from DC given as:

$\begin{matrix}{{\psi \left( {k,l} \right)} = {\sum\limits_{i = 1}^{N}{{\alpha (i)}\left\lbrack {{I\left( {i,j} \right)} \cdot {\cos \left( \frac{{\pi \left( {{2k} + 1} \right)}\left( {i - 1} \right)}{2N} \right)}} \right\rbrack}}} & (4)\end{matrix}$

where I is the N×N dimensional identity matrix and

${\alpha (i)} = \sqrt{\frac{1}{N}}$

when i=1 and

${\alpha (i)} = \sqrt{\frac{2}{N}}$

when i>1. A signal is k-sparse if it can be represented by a linearcombination of at most k cosine waveforms of variable frequencies, andthe other N−k coefficients are negligible. Since RIP guarantees signalrecovery for all k-sparse signals, CS implementations employing randommatrices are invariant to the image frequency content, as long as nottoo many frequency components are significant.

However, for natural images (i.e. images existing in the natural world),the vast majority of the signal power is carried by low frequencycomponents; high frequency components are in general very small in suchimages. Statistical data show that the signal power of natural imagesdecreases exponentially along the frequency axis. Intuitively, thispresence of primarily low frequency content leads to gradual changesamong neighboring pixel values. In other words, the average variance ofsignal power among neighboring pixels will be small. Taking advantage ofthis property, the present disclosure describes a method to perform CSmeasurement operations for image sensors which performs signalsummations only for a small number of neighboring pixels.

The operation of the disclosed method can be explained with reference tothe following example, which begins with reference to FIG. 2. Withoutlosing generality, assume that a CS measurement is conducted for a pixelcolumn containing 256 pixels and the compression rate R is 4. R isdefined as the ratio of the number of pixels over the number of CSmeasurements. Thus, 64 CS measurements are to be generated, which aredenoted by

₁,

₂,

3₆₄ in FIG. 2. To generate a single CS measurement, six neighboringpixels are added together, with an overlap of two pixels between twoneighboring summation groups. Except for

₂, the starting point for the next CS measurement will be four pixelsbeyond the first pixel of the previous CS measurement group, therebyyielding the overlap of two pixels between neighboring summation groups.

For example, in FIG. 2, the starting and ending pixel positions of eachsummation group are listed on the left side of the shaded summationgroup regions, and the 64 summation groups all refer back to the sameunderlying pixel column containing 256 pixels. The first CS measurement

₁ consists of the summation over the outputs of pixels 1-3 and 254-256.The next CS measurement

₂ will have an overlap of two pixels with

₁, and therefore contains the six pixels pixel 2-7. The third CSmeasurement

₃ will have an overlap of two pixels with

₂, and therefore contains the six pixels 6-11. Note that the second CSmeasurement

₂ has an overlap of two pixels with both

₁ and

₃.

Thereafter, the beginning pixel or pixel position of each subsequentmeasurement group is moved by 4 pixels to start the next CS measurement.For the convenience of discussion, let x be an N×1 vector formed bystringing together the pixel signals from an L×L pixel array in a columnafter column manner with N=L×L. Then, the pixel summations can bedescribed by the following equations:

1 m = ∑ i = m · L + 1 m · L + 3  x  ( i ) + ∑ i = ( m + 1 ) · L - 2 (m + 1 ) · L  x  ( i )   k m = ∑ i = m · L + 4 · k - 6 m · L + 4 ·k - 1  x  ( i ) , for   k > 1 ( 5 )

where m indicates the pixel column for which the CS measurement isperformed for 0≦m≦L−1.

In general, to generate M CS measurements for a pixel array containing Npixels, the size of summation groups should be

${\frac{N}{M} + {OL}},$

where OL represents the number of overlapping pixels between twoneighboring summation groups. In the example of FIG. 2, M equals 64, Nequals 256, and OL equals 2. Thus, the size of the summation groups is

${\frac{256}{64} + 2} = 6.$

As a guideline, OL is preferably selected as

$\frac{R}{2},$

if possible, recalling that R is the compression ratio. For given N, M,and OL values, the entries of measurement matrix φ can be determined as:

$\begin{matrix}{{\varphi \left( {i,j} \right)} = \left\{ \begin{matrix}1 & {{{{if}\mspace{14mu} 1} + \frac{\left( {i - 1} \right) \cdot N}{M}} \leq j \leq {\frac{i \cdot N}{M} + {OL}}} \\0 & {otherwise}\end{matrix} \right.} & (6)\end{matrix}$

for 1<i<M, 1<j<N. For i=1, M (i.e. the first row and the last row), thepattern can be slightly adjusted to meet the image size constraint,since the dimension of the frame may not be a multiple of

$\frac{N}{M}$

in general.

The above formula indicates that the majority of the elements of themeasurement matrix φ(i, j) are 0 and hence, such matrices are referredto as sparse measurement matrices. Note that the presently discussedsparse measurement matrices do not meet RIP constraints. However, theRIP condition represents a sufficient condition to guarantee therecovery of any k-sparse signals, but is not a necessary condition,especially when the signals of interest are mainly represented by lowfrequency components with respect to their sparse bases. Simulationswith various natural images show that sparse measurement matrices, asdefined above, achieve better performance than random matrices. 1000images from a publically available image benchmark database have beenused in the simulation study. The database contains eight imagecategories, each category covering different types of scenery (i.e.‘City Center Images’, ‘Forest Images’, ‘Mountain Images’, ‘StreetImages’, ‘Coast and Beach Images’, ‘Highway Images’, ‘Open CountryImages’, and ‘Tall Building Images’). The study examined the first 125images from each of the eight scenery categories. The peak signal tonoise ratios (PSNR) of the reconstructed images with the proposed CSmeasurement method are plotted (by the solid lines) in descending orderin FIG. 3, which contains one graph for each of the eight imagecategories. For comparison purposes, the PSNRs of the correspondingimages obtained using the conventional random measurement method arealso plotted using × markers in the figure. Note that higher PSNR valuesindicate better image qualities. As such, the advantage of using theproposed measurement technique with spare matrices over the conventionaltechnique of random matrices is evident, as it results in higher PSNRvalues for all the images except for three (one ‘Forest Image’, and two‘Tall Building Images’), for a rate of 0.3%. Even in these three outliercases, both methods lead to roughly the same image qualities.

CS Image Sensor Circuit Design

The present sparse measurement matrices advantageously enable a CS imagesensor circuit to be implemented in a very compact way, in a footprintthat previously was limited only to conventional image sensors. Inparticular, a CS image sensor circuit of the present disclosure can usecompact pixel cell designs and further does not require complex circuitsto perform CS measurement operations. For the purposes of clarity,discussion of the proposed CS image sensor circuit and techniques willcontinue to be made with reference to a 256×256 pixel array and acompression rate of 4, as discussed previously, although it isunderstood that the disclosed methods and techniques can be applied toother sizes of pixel arrays and different compression rates withoutdeparting from the scope of the disclosure. Additionally, it is notedthat the following discussion assumes that current-mode pixel cells areused in the pixel array, although voltage-mode pixel cells may also beused, in which pixel summations are carried out in terms of voltage orcharge summations.

FIG. 4 depicts a block diagram 400 of an exemplary CS image sensor ofthe present disclosure. As illustrated, pixel array 410 contains aplurality of pixels 412 and measures 256×256, for a total of 65,536pixels. The disclosed CS measurement method based on sparse measurementmatrices can conduct pixel summations in a row-by-row orcolumn-by-column manner, however, block diagram 400 is designed toconduct CS measurements in a column-by-column manner (i.e. bysequentially asserting column select lines CS₁, CS2 ₂, . . . , CS₂₅₆,labeled cumulatively as 422) in order to remain consistent with theexample presented by FIG. 2.

In order to conduct CS measurements in a column-by-column manner, theplurality of pixels 412 are communicatively coupled to a plurality ofpixel read lines 414 (also referred to as bit-lines) that are routedhorizontally within the pixel array 410. This is in contrast to theplurality of column select lines 422, which are routed vertically anddriven by a column selection circuit 420 outside of the pixel array 410.Row read operations can be performed in parallel for a given column, andas such, there is no row selection circuit.

During pixel read operations, the i^(th) column is selected by assertingCS, to a value of 1, thereby permitting all the pixels within the i^(th)column to be accessed. For example, to access the 1^(st) column, CS₁would be asserted to a value of 1. Within the selected column CS_(i),the output currents of the pixel cells that share the same bit-line areadded together at the bit-line and then are fed to the inputs of aplurality of current conveyors 430, which are labeled as CC₁,CC₂, . . ., CC₁₂₈ in the diagram 400.

As illustrated, the bit-lines 414 are shared by two pixel cells percolumn (i.e. two rows per bit-line). Note that each bit-line of theplurality of bit-lines 414 is labeled with the rows to which itcommunicatively couples. Beginning at the top of pixel array 410, thefirst bit-line is labeled “I_(R1)”. Immediately below the first bit-lineis a second bit-line labeled “I_(R2,3)”, indicating that it communicateswith only pixels in the second and third rows.

Additionally, the plurality of current conveyors 430 may also be dividedinto two categories. The first category of current conveyor has a singlecurrent output port (e.g. current conveyor CC₁), while the secondcategory of current conveyor has dual output ports (e.g. currentconveyor CC₂). The dual output port current conveyors are provided inorder to accommodate the overlaps between adjacent CS measurementgroups, as was previously illustrated in FIG. 2. The outputs of theplurality of current conveyors 430 are further summed according to themeasurement patterns before being fed to the inputs of a plurality ofdelta double sampling (DS) circuits 440, which are individually denotedby DS₁, DS₂, . . . , DS₆₄ in diagram 400. The current outputs of the DScircuits 440 are then amplified and converted to voltage signalsV_(out1), V_(out2), . . . , V_(out64) by trans-impedance amplifiers(TIAs) 450.

Thanks to its regularity, the CS measurement pattern is hardwired intothe design 400 of the image sensor. Recall that the measurement patterndiscussed with respect to FIG. 2 called for a first measurement group

₁ of pixels 254-256, 1-3, and note that DS_(i) receives as input thecurrent readings from I_(R254,255), I_(R1,256), and I_(R2,3). A secondmeasurement group

₂ of pixels 2-7 correlates to DS₂, which receives as input the currentreadings from I_(R2,3), I_(R4,5), and I_(R6,7), and so on. Thus, thedisclosed image sensor advantageously neither requires LFSRs for randombit generation nor uses complex pixel cells or complicated pixel signalrouting circuits that are otherwise required in order to support pixelrandom summation.

FIG. 5 presents a circuit diagram of various components presented inarchitectural diagram 400. In particular, illustrated are architecturesfor a pixel cell 512 (corresponding to one of the plurality of pixelcells 412), a current conveyor 530 (corresponding to one of theplurality of current conveyors 430), and a DS & TIA 540 (correspondingto one of the plurality of pairs of delta double sampling circuits 440and trans-impedance amplifiers 450). Pixel cell 512 is shown as a 3Tcurrent-mode active pixel cell, although it is understood that variousother current-mode and voltage-mode pixel cells may be employed withoutdeparting from the scope of the present disclosure.

Current conveyor 530 includes cascode current mirrors 536 a-c consistingof transistors M₅-M₁₀ and amplifier A₁, which forms a negative feedbackloop with transistor M₆ to keep the voltage of bit-line 514 at V_(b1).The first output branch 536 b of the current mirror, implemented bytransistors M₇ and M₈, is used by both types of current conveyors, andhence is drawn with a solid line. The second output branch 536 c of thecurrent mirror, implemented by transistors M₉ and M₁₀, is only neededfor dual output current conveyors (e.g. CC₂, CC₄, etc.) and hence isdrawn with a dotted line.

To keep transistor M₁ of the 3T pixel cell 512 in the linear region, thevoltage V_(b1) of bit-line 514 should be kept low. Meanwhile, thevoltage at the current mirror output is preferred to be relatively highdue to the consideration of signal swing headroom at TIA outputs. Ifpixel output ports were to be directly connected to the drain oftransistor M₅ in the current mirror input branch 536 a, this wouldpotentially result in a relatively large voltage difference between itsinput and output ports, which negatively affects current mirroraccuracy. To mitigate this problem, a diode connected transistor M₄ isinserted between the pixel bit-line 514 and the drain of transistor M₅for level shifting purposes.

To cope with transistor threshold variations across the pixel array,delta double sampling can be implemented in the design, as discussedpreviously with respect to the plurality of delta double samplers 440.Unlike conventional image sensors that conduct double sampling for eachpixel individually, the proposed design collectively performs doublesampling for the entire group of pixels to be summed in a single CSmeasurement. This does not diminish the benefit of double sampling,since the current errors caused by threshold variations are linear termsadded to the actual pixel signals. As illustrated inside thearchitecture of DS & TIA 540, transistors M₁₁-M₁₄ and capacitor C₁ forma current memory circuit that samples the summed pixel cell outputsafter the pixel integration period. Transistor M₁₂ is a sampling switchand transistor M₁₁, half the size of M₁₂, is provided to compensate forthe channel charge injection of M₁₂.

The cascode structure of transistors M₁₃ and M₁₄ keeps V_(DS13) at aconstant level, thereby helping improve the accuracy of the samplingcircuit. Immediately after finishing the above read operation, the pixelcells are reset and remain accessed. Then, the input of the DS circuitis the sum of the pixel output currents in a reset phase, which is thesecond sampling value of the double sampling operation. The two samplingvalues are naturally subtracted before being fed to the TIA via thetransmission gate consisting of transistors M₁₅ and M₁₆. The controlsignals CS₁, CS₂ for reading out the first two pixel columns aredepicted in FIG. 6, which indicates that the design takes two clockcycles to read a column and hence 512 cycles for the entire array.

FIG. 7 presents a comparison of the presently disclosed CS image sensordesign with existing designs across various aspects. It can be seen thatthe present CS image sensor design is the only design that usesconventional compact pixel cells and at the same time does not requirecomplex CS measurement circuits, which cover the functions ofmeasurement vector generation and selecting pixels to perform summationaccording to the measurement vectors. The present design furthereliminates the need of large LFSRs which are commonly used in allexisting CS image sensor circuits. The present design also does notrequire complicated column multiplexers since the pixel summation isrestricted to within the same column. These factors advantageouslybenefit the power and hardware efficiency of CS image sensor circuits.

Simulation Results

To validate the CS image sensor circuits based on the present sparsemeasurement matrices, two CS image sensors with compression rates of 4and 8 were designed. The pixel arrays of the two CS image sensors hadthe same size of 256×256 pixels and used the same 3T pixel cell design512 shown in FIG. 5. The circuit implementation of the CS image sensorwith compression rate 4 is illustrated in FIGS. 4 and 5. The sensor withcompression rate 8 was implemented similarly based on the proposedsparse measurement matrices, and sums 12 pixels in a single CSmeasurement such that there are overlaps of 4 pixels between neighboringsummation groups. Thus, for the CS image sensor with compression rate 8,four pixel rows share a bit-line, as opposed to the two pixel rows thatshare a bit-line for the CS image sensors with compression rate 4. Theimage sensor with a compression rate of 4 includes 128 currentconveyors, and 64 DS and TIA circuits. The image sensor with acompression rate of 8 includes 64 current conveyors, and 32 DS and TIAcircuits. The sensor circuits were designed using a 0.13 μm CMOStechnology and 1.5V power supply voltage. The transistor sizes and othercomponent values of the designs are shown in FIGS. 8 and 9.

Circuit simulations were conducted to obtain CS image sensor outputs forthe widely used benchmark images referred to as ‘Lena’ and ‘Cameraman’.The photocurrents in pixel cells were emulated by current sources in thecircuit simulations. After the circuit simulations, a Matlab l₁minimization package was used to reconstruct the images from the sensoroutputs. FIG. 10 shows the original images and the reconstructed images.The original images are placed on the left side of the figure, and thecircuit simulation reconstructed images are placed on the right side ofthe figure.

For comparison purposes, Matlab programs were also used to simulate theCS measurement operations implemented on the CS image sensors, where theMatlab simulations were based solely on the mathematics of the disclosedmeasurement method rather than any specific circuit implementation andtherefore yield an ideal case. The reconstructed images from the Matlabsimulations are given in the middle panel of the figure. The PSNRs ofthe reconstructed images are listed underneath the pictures. It showsthat the PSNR values of the reconstructed images from circuit simulationare very close to those obtained from Matlab simulation, indicating thatthe proposed CS image sensor circuits implement the proposed CSmeasurement method with high fidelity.

The power consumptions of the proposed CS image sensors were alsocompared with a conventional image sensor designed with the same CMOStechnology. These comparisons showed that the power consumption of thetwo CS image sensors was approximately ¼ and ⅛ of that of theconventional image sensor. The simulated energy per frame of the two CSimage sensors was also compared with existing CS image sensors, aslisted in the bottom row of FIG. 7, which indicates that both the R=4and the R=8 CS image sensors provide substantial power saving benefitsover existing CS image sensors. The low power benefit of the proposed CSimage sensors is mainly attributed to the simple CS measurementoperations associated with the present sparse measurement matrices.

The present disclosure describes a new CS measurement method that cansignificantly simplify CS image sensor implementation. Instead ofsumming randomly selected pixels, the present method performs pixelsummation for neighboring pixels within the same pixel column. Thepixels to be added together for generating a single CS measurementsample are said to belong a block, with neighboring blocks designed tohave certain overlaps and the first block designed to contain pixels atboth ends of the pixel column. This measurement method is illustrated bythe example shown in FIG. 2, in which the pixel column is assumed tocontain 256 pixels, for which 64 measurements (pixel summations) aremade.

The measurement matrix corresponding to the above CS measurementoperation is described by the general formula of Equation (6). Accordingto the formula, there are only a few non-zero elements in themeasurement matrix, such that the matrix is considered a sparsemeasurement matrix. Although the above example assumes 256 pixels in apixel column, the present measurement patterns can be easily modifiedfor different sizes of pixel column or for different compression ratiosby using the proposed general matrix formula.

The CS measurement method can be easily implemented on image sensors asshown in the designs of FIGS. 4 and 5. The disclosed CS image sensorsrequire only a small and simple digital circuit to control the CSmeasurement operations. The present method eliminates the need forcomplex and power-hungry circuit blocks that are otherwise required inthe existing CS image sensor circuits based on random measurementmatrices. Such complex circuit blocks include large linear feedbackshift registers (LFSRs) for generating pseudo random bit streams, shiftregister chains embedded into pixel arrays for pixel output control, andlarge multiplexer trees for routing signals from pixel cells todifferent summation circuits according to the random bit patterns. Theresultant hardware-efficient design helps reduce sensor powerconsumption and cost.

In some embodiments, the present method can be targeted towards imagesensors capturing natural images (images existing in the natural world).Statistical data show that the vast majority of the signal power ofnatural images is described by low frequency (or low index) coefficientsin their sparse representations with properly selected sparse basis. Thepresent method does not satisfy the RIP requirement, which is a verystrong requirement to guarantee robust signal recovery for all types ofsparse signals, regardless how the significant coefficients of thesparse signals are distributed in the sparse domains. Note that RIP is asufficient but not necessary condition for signal recovery. By takingadvantage of the aforementioned property of natural images, the presentmethod leads to better image quality compared to that obtained with theconventional random matrix based CS measurement method.

Existing CS image sensors known in the art are based on dense randommeasurement matrices. They all require complicated circuitimplementations, which degrade image sensor fill factors, powerefficiency as well as the scalability of CS techniques for highresolution image sensors. The present techniques can dramaticallysimplify CS image sensor circuits, helping reduce image sensor cost andpower consumption. The present method also improves the quality ofimages reconstructed from CS measurement samples.

While several particular embodiments of the present invention have beendescribed herein, it will be appreciated by those skilled in the artthat changes and modifications may be made thereto without departingfrom the invention in its broader aspects and as set forth in thefollowing claims.

What is claimed is:
 1. A method for compressive image sensing, themethod comprising: obtaining analog pixel data from a plurality of pixelcells; arranging the analog pixel data into a matrix form to generate apixel matrix; constructing a measurement matrix to divide the pixelmatrix into one or more summation groups, such that: each summationgroup comprises a summed value of consecutive pixel data; each summationgroup contains an equal amount of overlapping pixel data from a samenumber of pixel cells; and the measurement matrix is sparse; obtainingthe one or more summation groups by multiplying the measurement matrixwith the pixel matrix, wherein a number of summation groups is less thana number of pixels in the plurality of pixel cells; and analyzing theone or more summation groups to recover an image captured by theplurality of pixel cells.
 2. The method of claim 1, wherein the numberof summation groups is equal to the number of pixels divided by acompression rate.
 3. The method of claim 1, wherein the number ofsummation groups is equal to the number of rows of the measurementmatrix.
 4. The method of claim 1, wherein the measurement matrix andsummation groups are constructed in physical form on a circuit.
 5. Themethod of claim 1, wherein the plurality of pixel cells comprise animage sensor.
 6. The method of claim 1, wherein the consecutive pixeldata is obtained from the same number of pixel cells.
 7. The method ofclaim 1, wherein a number of pixels in the equal amount of overlappingpixel data contained in each summation group is between ¼ and ¾ of adesired compression rate of the compressive image sensing.
 8. The methodof claim 1, wherein a number of pixels contained in each summation groupis equal to the number of pixels in the plurality of pixel cells dividedby the number of summation groups, added with a number of pixels in theequal amount of overlapping pixel data contained in each summationgroup.
 9. A compressive sensing image system comprising: an array ofpixel cells for generating pixel data; a plurality of bit-lines, eachbit-line connected to one or more columns or one or more rows of pixelcells in the array; a measurement circuit coupled to the plurality ofbit-lines, the measurement circuit configured to receive one or moreoutputs from the pixel cells via the plurality of bit-lines and dividethe outputs into a plurality of summation groups, each summation groupcontaining an equal number of pixel cell outputs, and each summationgroup containing a same number of shared overlap pixel cell outputs; anda processor, the processor configured to receive as input the pluralityof summation groups from the measurement circuit and analyze theplurality of summation groups to recover an image captured by the arrayof pixel cells.
 10. The system of claim 9, wherein the measurementcircuit further comprises: a plurality of current conveyors, eachcurrent conveyor coupled to a bit-line; and a plurality of delta doublesampling circuits, each delta double sampling circuit coupled to theoutput of two or more current conveyors and generating as output asummation group.
 11. The measurement circuit of claim 10, wherein theplurality of current conveyors and delta double sampling circuits encodea sparse measurement matrix for application to the pixel data generatedby the array of pixel cells.
 12. The measurement circuit of claim 10,wherein the plurality of current conveyors and delta double samplingcircuits are arranged such that: the number of shared overlap pixel celloutputs is between ¼ and ¾ of a desired compression rate of thecompressive sensing image system.
 13. The measurement circuit of claim10, wherein the plurality of current conveyors and delta double samplingcircuits are arranged such that: the number of pixel cell outputscontained in each summation group is equal to the number of pixel cellsin the array divided by the number of summation groups, added with thesame number of shared overlap pixel cell outputs.
 14. The system ofclaim 10, wherein the number of CS measurement operations performed isless than the number of pixel cells in the pixel array and issubstantially equal to a number of summation groups in the plurality ofsummation groups.
 15. The system of claim 10, wherein the CS measurementcircuit further comprises a plurality of trans-impedance amplifiers eachcoupled to the output of a delta double sampling circuit.
 16. The systemof claim 9, wherein the array of pixel cells comprises current-modepixel cells.
 17. The system of claim 9, wherein the array of pixel cellscomprises voltage-mode pixel cells.
 18. The system of claim 9, whereinthe number of summation groups is less than the number of pixels in thearray of pixel cells.
 19. The system of claim 9, wherein the pluralityof bit-lines are connected to one or more columns of the array of pixelcells and the bit-lines are read in parallel by the CS measurementcircuit in a row-by-row fashion.
 20. The system of claim 9, wherein theplurality of bit-lines are connected to one or more rows of the array ofpixel cells and the bit-lines are read in parallel by the CS measurementcircuit in a column-by-column fashion.